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1.  Introduction 


The  US  Army  Research  Laboratory  (ARL)  Human  Sciences  Campaign  calls  out  the 
topic  of  Virtual/Mixed  and  Augmented  Reality  as  one  of  the  research  aims  of  2015- 
2035.  The  goal  is  to  use  human-machine  interaction  to  support  training  (Army 
Research  Laboratory  2014).  Augmented  reality  (AR)  is  a  type  of  virtual  environment. 
In  virtual  reality  (VR),  the  totality  of  the  environment  is  computer  generated.  In  AR, 
the  real  world  is  augmented  by  virtual  objects  or  entities  (Milgram  and  Kishino  1994; 
Azuma  1997).  AR  adds  to  the  real  world  environment,  where  VR  replaces  that 
environment.  The  Augmented  REality  Sandtable  (ARES)  is  uniquely  situated  to 
address  this  goal  using  inexpensive,  readily  available  commercial  technology.  ARES 
is  a  research  project  being  conducted  at  the  ARL  Human  Research  and  Engineering 
Directorate  (HRED)  Simulation  and  Training  Technology  Center  (STTC)  that 
combines  a  traditional  military  sand  table  with  a  Microsoft  Kinect  sensor  to  enable 
new  possibilities  for  terrain  visualization  and  learning  via  a  tangible  user  interface 
(Ishii  and  Ullmer  1997;  Ratti  et  al.  2004). 

2.  Definition  of  the  Problem 


Sand  table  exercises  (STEXs)  are  historically  recognized  as  an  effective  means  to 
conduct  tactical  training  with  an  emphasis  on  cognitive  skill  development  and 
tactical  decision  making  (Cohen  et  al.  1998;  Wildland  Fire  Lessons  Learned  Center 
2011).  The  ARES  research  testbed  seeks  to  maintain  the  positive  tangible  attributes 
of  the  traditional  sand  table  with  the  benefits  of  the  latest  in  commercial  off-the- 
shelf  (COTS)  digital  technologies.  One  area  of  research  for  ARES  is  the  tangibility 
aspects  of  simulation,  also  known  as  tangible  user  interfaces  (TUIs).  TUIs  are 
interfaces  in  which  digital  information  can  be  manipulated  using  physical  objects 
in  the  world,  such  as  your  hand  or  a  stylus  (Ishii  and  Ullmer  1997);  users  have  the 
capability  to  physically  interact  with  the  device  through  touch.  This  report  discusses 
some  of  the  existing  research  within  the  scope  of  TUIs  and  how  they  relate  to  the 
technology  of  ARES. 

The  desired  end  state  is  an  augmented  sand  table  platform  that  supports  a  variety  of 
research,  training,  and  operational  needs.  In  recent  years,  several  high-tech 
alternatives  to  the  traditional  sand  table  have  been  proposed,  such  as  multitouch 
surfaces  (Bortolaso  et  al.  2014),  3-dimensional  (3-D)  holographic  displays  (Mclntire 
et  al.  2014),  and  immersive  virtual  environments  (Qi  et  al.  2005).  However,  the  costs 
associated  with  developing,  fielding,  and  sustaining  such  systems  are  excessive, 
especially  when  compared  to  the  cost  of  a  traditional  sand  table.  This  cost  has  limited 
the  adoption  of  these  modem  technologies  (Sutton  2004). 
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3.  ARES  Components 


The  ARES  proof-of-concept  table  is  a  traditional  sand  table  filled  with  play  sand 
and  supplemented  with  low-cost  COTS  components,  which  are  shown  in  Fig.  1. 
They  include  the  following  equipment: 


A  commercial  projector  (--$900) 
Microsoft’s  Kinect  sensor  (-$200) 


A  COTS  laptop  (-$3,000) 


An  LCD  monitor  (-$400) 
Government-owned  ARES  software 
An  optional  web  camera  (-$100) 


Fig.  1  ARES  components 

The  Kinect  sensor  scans  the  surface  of  the  sand  and  detects  user  gestures  above  the 
sand  (Fig.  2).  Then  ARES  software  creates  a  map  of  the  sand  topography  and 
provides  that  to  client  applications.  This  allows  ARES  to  dynamically  respond  to 
the  changing  shape  of  the  sand  based  on  user  interaction  and  hand  gestures. 
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step  1:  Kinect  reads  sand  heights  Step  2:  Height  map  is  processed  Step  3:  Software  rendering  of 

by  ARES  software  height  data  is  projected  onto  the 

sand 


Fig.  2  Terrain  recognition  process 

4.  Potential  Research  Payoffs 

This  technology  may  provide  a  significant  return  on  investment  in  many  areas: 

.  Improved  spatial  awareness,  battlespace  visualization,  and  the  development 
of  a  common  operational  picture 

•  Increased  engagement,  retention,  and  collaboration  among  students  and 
operational  Warfighters 

•  Decreased  time  to  author  virtual  and  constructive  3-D  terrains  and  scenarios 

•  Joint/coalition  wargaming  and  mission  planning  through  networked  ARES 
tables 

5.  Relevance  to  the  Army 

With  an  investment  into  its  continued  research  and  development,  ARES  will 
support  an  array  of  Army  requirements.  Army  Warfighter  Outcomes  (Army 
Capabilities  Integration  Center  2012)  applicable  to  ARES  include  the  items  listed 
in  Eig.  3. 
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Training 


•  T-1:  Training 

•  T-2:  Reaiistic,  integrated  and  Mission  Command-centric  Training  Environment 

•  T-3:  Adaptive  Training  System 

•  T-4:  Contemporary  Leader  Skilis 

•  T-5:  Tailored  /  Adaptable  Learning  and  Training 

•  T-8:  Tailorable  Simulations  through  Gaming  Applications 

•  T-9:  Innovative  Learning  Models,  Strategies  and  Tools 


Mission  Command 


•  MC-2:  Create  Common  Situational  Understanding 

•  MC-3:  Mission  Command  On-The-Move 

•  MC-5:  Enable  Unified  Action  Partner  Collaboration 

•  MC-6:  Create,  Communicate,  and  Rehearse  Orders 

•  MC-10:  Airspace  Control  in  Unified  Action 


Human  Dimension 


•  HD-6:  Operational  Adaptability  and  Decision-Making 


Fig.  3  Army  Warfighter  Outcomes  supported  by  ARES  (Army  Capabilities  lutegratiou 
Ceuter  2012) 

6.  Related  Research 


There  are  related  past  research  efforts  to  ARES  in  academia,  industry,  and 
government.  Table  1  provides  a  comparison  between  related  past  research  and 
ARES. 


Table  1  Past  research  related  to  ARES 


First  author 
(year) 

Title 

Type  of  interface 

Projection 

Tangible 

user 

interface 

Low 

cost 

Human 

subjects 

testing 

Kirby 

(1995) 

NPSNET:  Software 
Requirements  for 
Implementation  of  a  Sand 
Table  in  the  Virtual 
Environment 

Completely  virtual 
interface  using 
Modular  Semi- 
Automated  Forces 

No 

No 

Unknown 

No 

Vaglia 

(1997) 

The  Virtual  Sand  Table 

Uses  a  Stylus  and 
Voice  Recognition 
to  interact  with 
interface 

Yes 

No 

Unknown 

No 

Alexander 

(2000) 

Visualisation  of 

Geographic  Data  in 

Virtual  Environments 

Projection,  Semi- 
Immersive  Display 
with  some 
interactivity 

Yes 

No 

Unknown 

Pilot 

Testing 

McGee 

(2001) 

Creating  Tangible 

Interfaces  by  Augmenting 
Physical 

Objects  with  Multimodal 
Language 

Uses  voice  and  pen 
recognition  as  well 
as  token  placement 
to  support  map 
based  planning 

Yes 

Yes 

Unknown 

Yes 
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Table  1 


Past  research  related  to  ARES  (continued) 


First  author 
(year) 

Title 

Type  of  interface 

Projection 

Tangible 

user 

interface 

Low 

cost 

Human 

subjects 

testing 

Wisher 

(2001) 

The  Virtual  Sand  Table: 
Intelligent  Tutoring  for 
Field  Artillery  Training 

Computer-based 
tutoring  system 
with  2-D  and  3-D 
perspective  views 

No 

No 

Unknown 

Yes 

Ishii 

(2002, 

2004) 

Bringing  Clay  and  Sand 
into  Digital  Design  — 
Continuous  Tangible  User 
Interfaces 

Projection  sand  / 
clay-based  tangible 
user  interface 

Yes 

Yes 

Depends 

on 

System 

Yes 

Reitmayr 

(2005) 

Localisation  and 

Interaction  for 

Augmented  Maps 

Projection-based 
map  augmentation 
using  a  Personal 
Digital  Assistant 
(PDA)  as  an 
interaction  device 

Yes 

Yes 

Unknown 

No 

Kobayashi 

(2006) 

Collaborative  Simulation 
Interface  for  Planning 
Disaster  Measures 

Projection-based 
tabletop  with 
manipulators 

Yes 

Yes 

Unknown 

Pilot 

Testing 

Couture 

(2008) 

GeoTUI:  A  Tangible  User 
Interface  for  Geoscience 

Projection-based 
tabletop  with 
multiple 
manipulators 

Yes 

Yes 

Unknown 

Yes 

Jung 

(2008) 

Virtual  Tactical  Map  with 
Tangible  Augmented 
Reality  Interface 

Tactical  map  with 
browser  and 
manipulators  / 
tangible  markers 

Yes 

Yes 

Unknown 

No 

Kalphat 

(2009) 

Tactical  Holograms  in 
Support  of  Mission 
Planning  and  Training 

Tested  terrain 
holographic  images 
with  Soldiers  to 
support  missions 

Yes 

No 

No 

Yes 

Hilliges 

(2009) 

Interactions  in  the  Air: 
Adding  Further  Depth  to 
Interactive  Tabletops 

Uses  sensors  and 
rear  projection  to 
allow  for 

manipulation  above 
the  tabletop 

Yes 

No 

No 

No 

Haihan 

(2010) 

Research  on  the 

Technology  of  Electronic 
Sand  Table  Based  on  GIS 

Virtual 

environment  sand 
table 

No 

No 

No 

No 

Martedi 

(2010) 

Foldable  Augmented 

Maps 

Using  a  Head- 
Mounted  Display 
(HMD)  and  maps 
with  intersection 
dots;  maps  can  be 
folded  and  still  use 
augmented  display 

No 

Yes 

No 

Yes 
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Table  1 


Past  research  related  to  ARES  (continued) 


First  author 
(year) 

Title 

Type  of  interface 

Projection 

Tangible 

user 

interface 

Low 

cost 

Human 

subjects 

testing 

Tateosian 

(2010) 

TanGeoMS:  Tangible 
Geospatial  Modeling 
System 

Tangible  geospatial 
modeling  system 
which  uses  a  laser 

scanner 

Yes 

Yes 

No 

No 

Schneider 

(2011) 

Benefits  of  a  Tangible 
Interface  for 

Collaborative  Learning 
and  Interaction 

Projection-based 
tangible  user 
interface  to 
determine  object 
layout 

Yes 

Yes 

Unknown 

Yes 

Zhang 

(2011) 

AR  Sand  Table  with 
VSTAR  System 

Augmented  Reality 
Sand  table  with 
viewer  and  an 
attached  camera 

No 

No 

No 

No 

Harshitha 

(2013) 

HCI  Using  Hand  Gesture 
Recognition  for  Digital 
Sand  Model 

Uses  projections, 
mirrors,  cameras 
and  terrain  with 
gesture  control 

Yes 

Yes 

Unknown 

No 

Schneider 

(2013) 

Preparing  for  Future 
Learning  with  a  Tangible 
User  Interface:  The  Case 
of  Neuroscience 

Underneath 

Tabletop 

Projection, 

Tangible  User 
Interface 

Yes 

Yes 

Unknown 

Yes 

Bortolaso 

(2014) 

Design  of  a  Multi-Touch 
Tabletop  for  Simulation- 
Based  Training 

Tabletop  Command 
and  Control 

Tangible  User 
Interface 

No 

Yes 

No 

Yes 

Jianping 

(2014) 

An  Application 
Development 

Environment  for 
Collaborative  Training 

Sand  Table 

Multi  touch 
tabletop  for 
ecommerce 

No 

Yes 

No 

1  1 

No 

Petrasova 

(2014) 

GIS-Based  Environmental 
Modeling  with  Tangible 
Interaction  and  Dynamic 
Visualization 

Follow-up  to 
Tateosian  (2010): 
Tangible  interface 
using  Kinect  and 
sand  to  form 
terrains 

Yes 

Yes 

Yes 

No 

Xiaofeng 

(2014) 

Making  a  Virtual  Sand 
Table  Based  on  Unity  3D 
Technique 

Full  Virtual 
Environment 

No 

No 

Unknown 

No 

Zhao 

(2014) 

Research  and  Design  on 
Power  SUPPLY 
NETWORK  Sand  Table 
Exercise  System 

Virtual  Display 
with  Elevation 

Maps  for  Power 

Grids  using  sand 
table  principles 

No 

No 

Unknown 

No 

Ma  (2015) 

Using  a  Tangible  Versus  a 
Multi-touch  Graphical 

User  Interface  to  Support 
Data  Exploration  at  a 
Museum  Exhibit 

Tabletop  Tangible 
User  Interface  with 
Manipulators 

No 

Yes 

No 

Yes 
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7.  ARES  Current  Demonstration  Capabilities 


ARES  currently  has  the  following  capabilities: 

.  Projection  of  Images,  Maps,  and  Video  From  Any  Source  onto  the  Sand 

ARES  can  be  logged  into  a  VBS3  (Virtual  Battlespace  3)  or  OneSAF  (One 
Semi- Automated  Forces)  scenario,  for  example,  and  display  a  top-down 
“instructor’s  view”  of  the  scenario  in  real  time. 

•  Export  of  the  User-Shaped  Sand  as  a  3-D  Terrain  File 

Created  terrains  (and  potentially,  scenarios)  can  be  imported  into  a  variety 
of  simulation  applications,  such  as  VBS3  or  OneSAF. 

.  Placement  and  Labeling  of  Units 

A  subset  of  Military  Standard  (MIF-STD)  2525C  military  symbols  and  icons 
and  tactical  graphics  can  be  created,  labeled,  and  placed  for  mission  planning; 
these  plans  (“scenarios”)  can  then  be  saved  for  later  reuse  (Fig.  4). 


Fig.  4  Using  a  tablet  to  place  units  on  ARES 
.  Assisted  Terrain  Correlation 

Through  visual  aids  (e.g.,  color  schemes  and  eontour  lines),  ARES  provides 
users  with  basic  guidance  in  shaping  the  sand  to  match  a  previously  saved 
3-D  terrain. 

•  Universal  Serial  Bus  Human  Interface  Device 

Capability  for  ARES  to  be  viewed  as  a  standard  interface  device  (like  a 
mouse  or  keyboard)  so  that  users  can  plug  ARES  into  a  simulation  and  use 
hand  tracking  to  navigate  menus  as  if  their  hand  were  the  mouse. 
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.  Hand/Gesture  Tracking 

Detects  and  tracks  the  presence  of  a  hand,  or  hands,  above  the  sand  and 
allows  for  software  responses  to  be  linked  to  where  a  user  points. 

•  Video  Teleconference 

An  inexpensive  commercial  web  camera  and  integrated  software  allow 
distributed  users  to  communicate  and  collaborate  in  real  time.  Through  the 
use  of  the  integrated  color  camera  on  the  Kinect  sensor,  users  are  also 
provided  a  top-down  view  of  the  collaborator’s  table. 

.  Augmented  Reality  (AR)  Peripherals 

Layers  of  data  can  be  displayed  through  AR  goggles  or  AR  apps  on  a  tablet. 
For  example,  with  collaboration  from  Marine  Corps  Systems  Command, 
helicopters  that  hover  and  fly  routes  above  the  sand  have  been  demonstrated 
as  an  early  concept.  When  the  Soldier  Training  Enhancement  Package 
(STEP)  3-D,  another  STTC  research  project,  is  incorporated,  3-D  buildings 
and  vehicles  pop  up  from  the  terrain  (Pig.  5). 


Fig.  5  Augmented  reality  vehicles  and  buildings  on  ARES 
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.  Contour  Lines 

The  system  can  display  terrain  features  and  hypsometric  displays  (terrain 
contour  lines)  to  mimic  topographic  maps. 

.  Line  of  Sight  (LOS) 

The  system  allows  for  the  dynamic  display  of  a  selected  unit’s  LOS  either 
based  on  the  sand’s  topology  or  on  synthetic  environment  data  provided  in 
the  Layered  Terrain  Format  (LTF). 

8.  ARES  Ongoing  Research  and  Development 

The  following  ARES  research  and  development  efforts  are  ongoing; 

.  Effects  of  Different  Mediums 

Other  mediums  (e.g.,  different  types  of  sand,  clay,  magnets,  3-D  printed 
relief  maps)  offer  different  benefits,  such  as  the  ability  to  represent  vertical 
surfaces.  The  effects  of  using  these  alternatives  (realism,  accuracy, 
immersion)  are  being  investigated  to  understand  how  the  selection  of 
medium  changes  human  performance  factors. 

.  Expansion  of  Unit  Libraries 

More  of  the  MIL-STD-2525C  unit  iconography  is  being  incorporated  to 
support  wider  applications. 

.  Shared  Table  States 

Networked  tables  will  be  able  to  share/compare  topography  data  and  unit 
placement  to  support  distributed  collaboration  (training,  planning, 
operational  support)  across  the  room  or  across  the  globe. 

.  Rapid  Authoring  of  Training  Content 

Analysis  and  design  of  a  user  interface  allowing  end  users  to  create  their 
own  instructional  content  on  ARES,  including  the  ability  for  learners  to 
answer  questions  naturally  by  pointing  to  locations  or  units  on  the  sand. 

•  Operational  Unit  Research  and  Evaluation 

Coordination  has  begun  to  conduct  experiments  on  usability,  effectiveness, 
transfer,  and  operational  impact  with  operational  military  units,  such  as  the 
3rd  Infantry  Division  at  Ft.  Benning. 

.  Assisted  Terrain  Correlation 

The  existing  base  functionality  will  be  examined  for  enhancements  to  speed 
and  usability. 


9 


•  Publishing  of  Architecture 

The  ARES  architecture  and  Application  Programing  Interface  are  being 
published  for  community  feedback  and  development  of  end-user 
applications. 

.  ARES’  Relationship  to  Affect/Engagement 

Research  is  planned  to  assess  the  affective  change  caused  by  the  interaction 
with  ARES. 

•  Sample  Application:  Tank  Combat  Game 

A  simple  tank  combat  game  (Eig.  6)  is  being  developed  to  evaluate, 
demonstrate,  and  document  the  capability  for  end  users  to  easily  develop 
their  own  custom  apps  that  leverage  the  ARES  platform. 


Fig.  6  Tank  combat  game  application 
.  ARES  Scalability 

Based  on  user  requirements,  a  squad-sized  version  of  ARES  (7  x  4  ft)  is 
under  construction,  and  future  scalability  needs  are  being  evaluated. 

.  After- Action  Review  Tools 

Develop  and  evaluate  tools  to  support  after-action  reviews  of  simulations  or 
live  training  events. 

9.  Desired  Future  Research  Questions  and  Capabilities 
Development 

The  following  list  consists  of  capabilities  we  aim  to  develop  and  various  research 
questions: 

.  Dynamic  Projection  Mapping 

Support  for  complex  surfaces  so  projected  images  do  not  look  “warped”  on 
sand  topology. 
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.  Connection  to  Geographic  Databases 

Research  is  needed  to  determine  the  best  method  of  providing  cues  to  the 
user  that  would  prompt  them  to  shape  the  sand  to  correlate  to  a 
geographically  specific  area  of  operation. 

.  Hand/Finger  Tracking  for  Unit  Placement  and  Command 

Incorporating  tracking  capabilities  that  will  provide  users  the  ability  to 
select  and  move  objects  by  pointing  at  locations  on  the  sand.  Refinement  of 
hand  tracking  sensitivity  is  needed  to  support  the  development  of  a  gesture 
set  that  can  be  used  to  intuitively  command  and  control  units. 

.  Military  Map  Features 

Inclusion  of  military  overlay  data,  such  as  grid  lines,  military  grid  reference 
system  coordinates,  tactical  landmarks,  and  elevation  notation. 

.  Augmented  Reality  to  Improve  Data  Visualization  and  Decrease 
Cognitive  Load 

Technology  that  will  allow  certain  information  to  “pop”  up  off  the  sand 
when  viewed  through  a  mobile  device  or  AR  goggles.  For  example,  users 
can  see  communications  networks  or  tall  structures  that  may  impede  air 
support  above  the  topography. 

.  Affective  Benefits  of  ARES  as  a  TUI  in  a  Military  Context 

Research  study  to  determine  if  ARES  users  see  an  increase  in  engagement, 
collaboration,  and  interactions  and  what  effect  that  has  on  learning  and 
performance  outcomes. 

.  Incorporation  into  Live  Events 

Integrated  with  Force  XXI  Battle  Command  Brigade  and  Below  (Army)  / 
Instrumented-Tactical  Engagement  Simulation  System  (Marines) 
technology  to  import  and  display  unit  locations  and  states  in  real  time  on 
ARES  for  improved  understanding  of  the  operational  picture. 

.  Man-Portable  Form  Factor 

Eeverage  COTS  mobile  device  sensors  and  mini-projection  technologies  to 
provide  a  reduced-scale  version  that  can  be  carried  by  a  user  and  projected 
onto  any  surface  (Fig.  7). 
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Fig.  7  Concept  of  mobile  version  of  ARES 

.  Can  ARES  be  used  for  more  efficient  terrain  and  scenario  generation? 

The  capability  to  author  and  export  3-D  terrains  and  scenario  settings  in 
standardized  formats  will  be  developed  and  tested.  This  will  allow  subject 
matter  experts  (SMEs)  and  instructors  to  create  content  on  ARES  and  export 
the  content  to  other  simulation  and  game  applications  (OneSAF,  VBS3, 
etc.)  more  efficiently  than  via  traditional  methods  and  without  requiring  the 
highly  specialized  skills  of  platform-specific  scenario  authors  and  3-D 
artists. 

.  Can  distributed  users  collaborate  more  effectively? 

The  distributed,  accurate  common  operational  picture  on  ARES  will  be  used 
as  a  testbed  to  determine  if  joint  and  coalition  partners  can  collaborate  on 
tasks,  such  as  exercise  planning  and  tactical  war-gaming  in  a  more  intuitive 
and  cost-effective  manner. 

10.  ARES  Research  Methodology 

ARES  research  began  with  a  literature  review  of  current  sand  table  use,  specifically 
STEX  (i.e.,  how  they  have  been  used  historically  and  best  practices  for  their  use); 
studies  on  related  topics,  such  as  the  benefits  of  tangible  user  interfaces;  and  the 
effectiveness  of  virtual  sand  tables  and  similar  systems.  A  market  survey  was  also 
done  to  discover  the  state  of  the  art  in  related  technologies. 

Next,  an  analysis  was  conducted  to  identify  potential  use  cases,  military  end  users 
and  stakeholders,  and  core  functionalities  and  features  that  users  would  require  of 
ARES.  This  analysis  included  trips  to  the  199th  Infantry  Brigade  at  Ft.  Benning  and 
to  The  Basic  School  at  Quantico  Marine  Corps  Base  to  speak  with  and  observe 
SMEs.  SMEs  from  both  services  were  enthusiastic  about  ARES  and  provided  use 
cases  where  ARES  is  capable  of  enhancing  training  throughout  their  leadership 


12 


courses.  The  findings  of  this  analysis  were  used  to  define  the  initial  capabilities  and 
scope  of  the  ARES  proof-of-concept  table  along  with  identifying  the  required 
research. 

Using  only  mission  funding  internal  to  STTC,  we  eonstructed  an  ARES  proof-of- 
eoneept  table  to  determine  feasibility  to  support  researeh  studies  and  to  demonstrate 
the  concept  to  potential  stakeholders,  collaborators,  and  users  for  requirements 
definition. 

11.  ARES  Research  Experiments 

Several  studies  using  ARES  are  being  eondueted  to  define  the  benefits  of  sand  table 
interactions: 

•  Charles  Amburn,  principal  investigator  for  the  ARES  project,  is  conducting 
a  study  with  2  main  objectives:  1)  to  determine  the  extent  to  which  ARES 
is  capable  of  signifieantly  improving  spatial  knowledge  aequisition  above 
and  beyond  existent  training  mediums  (paper  map  and  2-D  display  of  3-D 
rendered  objects)  and  2)  to  provide  a  foundation  for  establishment  of  a 
learner-centric  environment  using  novel  technology.  Essentially,  the 
hypothesis  is  that  users  viewing  maps  on  the  sand  are  more  effective  and 
efficient  than  those  using  2-D  printed  maps  or  even  3-D  digital 
representations  presented  on  a  2-D  plane  (e.g.,  an  ECD  monitor  or  tablet). 

.  Miehael  Boyce  is  currently  condueting  a  study,  “The  Effect  of  Topography 
on  Teaming  Tactical  Maneuvers”,  wherein  he  is  assessing  the  effects  of 
ARES’  3-D  surface  on  learning  taetieal  military  knowledge.  The 
Department  of  Military  Instmction  at  the  United  States  Military  Aeademy 
(USMA)  is  assisting  with  the  pilot  study,  which  is  planned  to  become  a  full 
study  to  be  eondueted  at  USMA  in  FY16. 

12.  ARES  as  a  Service 


ARES  is  built  on  a  service-oriented  arehiteeture  that  is  modular  and  extensible, 
allowing  for  customization  to  meet  individual  user  needs.  Now  that  the  ARES 
software  and  interface  protoeols  are  published,  other  users  (e.g..  Army,  Department 
of  Defense  [DOD],  and  other  government  ageneies)  ean  leverage  the  teehnology  in 
new  ways.  This  flexible  approach  and  open  interface  allows  for  2  primary  benefits: 

•  ARES  can  be  updated  easily  and  cost  effectively,  taking  advantage  of  the 
latest  advances  in  sensor,  projeetor,  and  augmented  reality  teehnologies. 
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ARES  becomes  a  platform,  similar  to  iOS  or  Android,  where  users  will 
create  their  own  apps  to  support  their  individual  needs. 


Figure  8  illustrates  the  goal  of  extending  core  capabilities  with  applications  and 
services  to  support  a  myriad  of  training  and  operational  tasks. 
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Fig.  8  An  open  ecosystem  of  cooperating  services  and  capabilities  that  can  be  combined  to 

meet  end-nser  needs 


13.  ARES  Reuse  of  Existing  Technologies 


ARES  uses  an  open  systems  architecture  that  is  modular  and  extensible.  It  already 
successfully  leverages  several  existing  technologies  developed  by  ARE.  Table  2 
outlines  reused  technologies. 
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Table  2  ARES  reuse  of  existing  technologies 


Integrated  technology 

Technology  description 

Specific  integration  with 
ARES 

Rapid  Unified  Generation  of 

Urban  Databases  (RUGUD) 
http://www.rapidterrain.com 

RUGUD  is  a  government 
off-the-shelf  data  processing 
framework  that  can  assist  in 
the  conversion  of  terrain 
data  into  alternate  formats  to 
be  used  by  other 
applications. 

RUGUD  is  used  with 

ARES  as  a  mechanism  to 
translate  other  terrain 
formats  into  formats  that 
ARES  can  understand. 

Layered  Terrain  Format  (LTF) 
(Peele  etal.  2011) 

LTF  is  a  high -resolution  3- 
D  terrain  representation 
format  that  can  be  rendered 
on  hardware  and  software 
constraint  devices,  such  as  a 
hand-held. 

LTF  is  used  with  ARES  as 
a  part  of  its  tablet-based 
interaction  platform 
facilitating  mission 
planning  and  LOS. 

Tactical  Terrain  Analysis  (TTA) 
App 

(Borkman  et  al.  2010) 

The  TTA  app  provides  a 
line  of  sight  (LOS)  service 
that  can  provide  a  fan 
representing  the  field  of 
view  at  a  given  location, 
incorporating  occlusion  for 
areas  that  are  blocked 
because  of  obstructions. 

ARES  uses  the  TTA  app 
functionality  to  determine 
LOS  given  a  particular 
terrain  topology,  such  that  it 
changes  dynamically  as  the 
sand  is  manipulated. 

Soldier  Training  Enhancement 
Package  (STEP)  3D  App 
(Roberts  and  Chen  2009) 

The  STEP  3-D  App  is  an 
augmented  reality  tool  that 
allows  for  buildings  to 
appear  in  3-D  when  viewed 
through  a  tablet. 

STEP  3-D  is  used  on  ARES 
in  conjunction  with 
recognition  markers  to 
cause  buildings  to  show  3- 
D  height  on  the  sand. 

Upcoming  integrations 

Technology  description 

Specific  integration  with 
ARES 

Generalized  Intelligent 

Framework  for  Tutoring  (GIFT) 
(Sottilare  et  al.  2012) 

GIFT  is  a  modular,  service- 
oriented  architecture 
developed  to  assist  in  the 
authoring  and  development 
of  Intelligent  Tutoring 
Systems. 

GIFT  will  provide  ARES 
with  the  capability  to  author 
adaptive  training 
experiences  that  can  adjust 
to  the  needs  of  an 
individual  learner. 

Web-Based  Military  Scenario 
Development  Environment 
(WebMSDE) 

(Marshall  et  al.  2013) 

WebMSDE  is  a  scenario 
generation  tool  that  can  be 
executed  within  a  browser. 

It  has  a  robust  user  interface 
and  can  efficiently  generate 
scenarios  in  a  wide  variety 
of  formats. 

WebMSDE  may  provide 
ARES  with  distributed 
OneSAF  scenario 
generation  capability  as 
well  as  access  to  its  existing 
scenario  libraries. 
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14.  Evaluations 


Several  instances  of  the  ARES  proof  of  concept  have  been  strategically  coordinated 
with  stakeholders  for  continued  evaluation  of  current  capabilities  and  the 
development  of  future  research  requirements.  For  example,  the  Office  of  Naval 
Research  (ONR)  has  provided  funding  for  ARES  to  be  evaluated  in  Quantico  by  the 
Marine  Corps’  The  Basic  School.  Feedback  has  been  positive,  and  they  expressed 
eagerness  to  run  force-on-force  training  with  networked  ARES  tables.  Additional 
desired  capabilities  have  been  submitted  through  ONR,  including  the  intent  for 
ARES  to  scale  up  to  cover  the  large  16-ft  battalion-sized  sand  tables  they  use. 

Per  a  March  2014  Memorandum  of  Understanding  (MOU)  with  USMA  at  West 
Point,  STTC  provided  an  ARES  prototype  to  the  Engineering  Psychology  Program 
of  the  Department  of  Behavioral  Sciences  and  Eeadership  at  USMA  where  first  class 
cadets  (seniors)  in  the  Engineering  Psychology  program  have  conducted  initial 
usability,  workspace,  and  workload  assessments  as  part  of  their  final  human  factors 
design  course.  ARES  is  also  being  used  by  cadets  learning  to  conduct  Operational 
Order  (OPORD)  briefs  as  a  part  of  their  military  instruction  curriculum  (Fig.  9). 


Fig.  9  Cadet  at  West  Point  performing  OPORD  briefing  on  ARES  (image  conrtesy  of  the 
Engineering  Psychology  Program,  Department  of  Behavioral  Science  and  Leadership,  United 
States  Military  Academy,  West  Point) 

Under  a  different  collaboration  between  ARE  and  the  211th  Regional  Training 
Institute,  the  Florida  National  Guard  has  provided  funding  for  an  ARES  prototype 
to  be  delivered  to  Camp  Blanding  for  evaluation  and  research  collaboration  by  first- 
quarter  2016. 

STTC  is  currently  working  with  the  Rapid  Equipping  Force  to  place  one  or  more 
ARES  prototypes  within  operational  units  to  conduct  evaluations  and  experiments 
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related  to  effeetiveness  and  efficieney.  The  initial  insertion  is  being  eoordinated 
with  the  3rd  Infantry  Division  at  Ft.  Benning. 

Work  is  also  being  done  under  the  seope  of  joint/coalition  training  research  at  STTC 
to  integrate  a  distributed  scenario  authoring  tool  (WebMSDE)  and  to  improve  the 
distributed  collaboration  capabilities  in  order  to  be  evaluated  ARES  with  joint/ 
coalition  partners. 

15.  Demonstrations  and  Key  Leader  Engagements 

In  seeking  additional  stakeholders  and  collaborators,  the  ARES  project  has  been  a 
part  of  several  hundred  demonstrations  and  discussions  with  DOD  senior 
leadership,  members  of  Congress,  representatives  from  other  interested  agencies, 
and  foreign  militaries. 

ARES  has  been  briefed  and  discussed  at  the  following  locations; 

•  Simulation  and  Training  Technology  Center 

.  Program  Executive  Office  for  Simulation,  Training,  and  Instrumentation 
(PEO  STRI) 

•  1 43rd  Expeditionary  Sustainment  Command 

.  2014  Interservice/Industry  Training,  Simulation,  and  Education  Conference 

•  Modem  Day  Marine  2014 
.  Navy  Euture  Forces  2015 

•  Human  Factors  Engineering  Technical  Advisory  Group  in  May  2015 

.  2015  Association  of  the  United  States  Army  Global  Force  Symposium  and 

Exposition  (Fig.  10) 

Army  leaders  who  have  recently  provided  extremely  positive  feedback  and 
guidance  include  Major  General  Wharton  (Commanding  General,  US  Army 
Research,  Development,  and  Engineering  Command),  Major  General  Maddux 
(PEO  STRI),  General  Perkins  (Commanding  General,  US  Army  Training  and 
Doctrine  Command),  General  Via  (US  Army  Materiel  Command  Commanding 
General)  and  General  Odiemo  (Chief  of  Staff  of  the  Army). 
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Fig.  10  ARES  being  briefed  to  General  Via  at  AUSA  2015 

16.  The  Way  Forward 

The  Science  and  Technology  activity  at  ART  HRED  STTC  will  use  mission  funds 
to  cover  some  of  the  initial  research  required,  and  STTC  will  make  some  form  of  it 
available  to  selected  units  to  conduct  evaluations  and/or  experiments.  To  fully 
evaluate  its  potential,  ARL  HRED  STTC  needs  support  in  defining  a  US  Army 
Training  and  Doctrine  Command  (TRADOC)  requirement  and  a  transition  path. 
Although  some  of  the  basic  initial  research  is  currently  underway,  funds  must  be 
injected  from  sources  outside  of  STTC  to  further  the  research  and  enable  ARES  to 
reach  Warfighter  needs.  Once  a  requirement  and  funding  are  in  place,  an  acquisition 
strategy  will  be  set  forth  to  transition  the  technology  to  a  program  of  record  for 
fielding  and  sustainment. 

ARES  is  a  rare  project  that  sparks  the  imagination  in  almost  everyone  who  interacts 
with  it.  There  is  something  intangible  about  the  tactile  medium  that  draws  them  in, 
makes  them  comfortable,  and  makes  them  want  to  explore  and  create.  With  modern, 
low-cost  technologies,  the  nature  and  benefits  of  these  interactions  can  be  studied 
and  leveraged  in  ARES  and  other  Army  projects.  What  if  all  training  made  you  feel 
engrossed  and  energized?  What  if  all  mission  planning  tools  encouraged  creative 
thinking?  And  what  if  all  modeling  and  simulation  systems  felt  intuitive,  logical, 
and  natural? 

Given  the  practical  limitation  on  how  much  information  a  Warfighter  can 
effectively  handle,  further  research  in  the  realm  of  presenting  Warfighters  with 
tangible,  intuitive  interfaces  that  allow  for  them  to  reveal  layers  of  data  based  on 
specific  mission  requirements  is  an  imperative.  The  principles  that  we  learn  from 
ARES  can  be  applied  to  create  new  forms  of  data  visualization  and  computer 
interaction.  Continued  ARES  research  will  enable  a  set  of  standards  for  ambient 
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computing — ^where  the  room  around  you  is  the  display  (not  just  your  flat  eomputer 
monitor)  and  the  tools  around  you  (your  hands,  the  pencil,  and  coffee  mug  on  your 
desk)  will  be  used  to  explore  the  data  you  need  to  fully  understand  the  mission  and 
share  that  understanding  effeetively. 
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3-D 

3 -dimensional 

AR 

augmented  reality 

ARES 

Augmented  REality  Sandtable 

ARE 

US  Army  Research  Laboratory 

COTS 

commercial  off-the-shelf 

DOD 

Department  of  Defense 

GIET 

Generalized  Intelligent  Framework  for  Tutoring 

HMD 

Head-Mounted  Display 

HRED 

Human  Research  and  Engineering  Directorate 

LOS 

line  of  sight 

LTF 

Layered  Terrain  Format 

MOU 

Memorandum  of  Understanding 

OneSAF 

One  Semi- Automated  Forces 

ONR 

Office  of  Naval  Research 

OPORD 

Operational  Order 

PDA 

Personal  Digital  Assistant 

PEO  STRI 

Program  Executive  Office  for  Simulation,  Training, 
Instrumentation 

RUGUD 

Rapid  Unified  Generation  of  Urban  Databases 

SME 

subject  matter  expert 

STEP 

Soldier  Training  Enhancement  Package 

STEX 

sand  table  exercises 

STTC 

Simulation  and  Training  Technology  Center 

TRADOC 

US  Army  Training  and  Doctrine  Command 

TTA 

Tactical  Terrain  Analysis 

TUI 

tangible  user  interface 

USMA 

United  States  Military  Academy 
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VBS3 

VR 

WebMSDE 


Virtual  Battlespace  3 
virtual  reality 

Web-Based  Military  Seenario  Development  Environment 
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